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Abstract 

Clinical drug trials are often conducted in selective patient populations, with relatively small numbers of patients, 
and a short duration of follow-up. Observational studies are therefore important for collecting additional 
information on adverse drug events (ADEs). Currently, there is no guidance regarding the methodology for 
measuring ADEs in such studies. Our aim was to evaluate whether the methodology used to assess non-serious 
ADEs in observational studies is adequate for detecting these ADEs, and for addressing limitations from clinical 
trials in patients with type 2 diabetes mellitus. We systematically searched MEDLINE and EMBASE for observational 
studies reporting non-serious ADEs (1999-2008). Methods to assess ADEs were classified as: 1) medical record 
review; 2) surveillance by health care professionals (HCP); 3) patient survey; 4) administrative data; 5) laboratory/ 
clinical values; 6) not specified. We compared the range of ADEs identified, number and selection of patients 
included, and duration of follow-up. Out of 10,125 publications, 68 studies met our inclusion criteria. The most 
common methods were based on laboratory/clinical values (n = 25) and medical record review (n = 18). Solicited 
surveillance by HCP (n = 17) revealed the largest diversity of ADEs. Patient surveys (n = 15) focused mostly on 
hypoglycaemia and gastrointestinal ADEs, laboratory values based studies on hepatic and metabolic ADEs, and 
administrative database studies (n = 5) on cardiovascular ADEs. Four studies presented ADEs that were identified 
with the use of more than one method. The patient population was restricted to a lower risk population in 19% of 
the studies. Less than one third of the studies exceeded pre-approval regulatory requirements for sample size and 
duration of follow-up. We conclude that the current assessment of ADEs is hampered by the choice of methods. 
Many observational studies rely on methods that are inadequate for identifying all possible ADEs. Patient-reported 
outcomes and combinations of methods are underutilized. Furthermore, while observational studies often include 
unselective patient populations, many do not adequately address other limitations of pre-approval trials. This 
implies that these studies will not provide sufficient information about ADEs to clinicians and patients. Better 
protocols are needed on how to assess adverse drug events not only in clinical trials but also in observational 
studies. 
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HEALTH AND QUALITY 
OF LIFE OUTCOMES 



Introduction 

Medication safety assessment during the pre-approval 
regulatory phase is known to have limitations. Pre- 
approval clinical trials are often conducted in selective 
patient populations, with relatively small numbers of 
patients, and a short duration of follow-up [1,2]. Because 
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of these limitations, several systems have been developed 
to monitor drug safety after marketing, including spon- 
taneous reporting systems and risk management plans. 
Such safety assessment focuses primarily on detection of 
serious adverse drug events (ADEs) [3]. Little attention 
is given to the assessment of symptomatic or non-life- 
threatening ADEs, while the proportion of such ADEs is 
relatively common [4,5]. Symptomatic ADEs may affect 
patients' quality of life and adherence to treatment, and 
thereby the risk-benefit ratio of a drug. 



O© 201 1 Hakobyan et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative 
BIoIVIGCI C6ntr3l commons Attribution License (http://creativecommons.0rg/licenses/by/2.O), which permits unrestricted use, distribution, and 
reproduction in any medium, provided the original work is properly cited. 



Hakobyan et al. Health and Quality of Life Outcomes 201 1, 9:83 
http://www.hqlo.eom/content/9/1/83 



Page 2 of 9 



Post-marketing observational studies are considered 
important to get more information on ADEs occurring 
in patient populations actually using the drugs [2,6,7]. 
This additional value, however, will only be achieved 
when the methodology used in such studies allows for 
adequate capturing of non-serious ADEs in an unrest- 
ricted population. The use of different methods for 
assessing ADEs, such as spontaneous and solicited 
reporting, medical record review, and patient surveys, 
may lead to differences in observed ADEs [8,9]. No gui- 
dance exists regarding the methods to be used for mea- 
suring ADEs in post-marketing studies [10-13]. 

Our aim was to evaluate the current methodology for 
assessing non-serious ADEs in observational studies, 
using oral antihyperglycemic drugs (OAD) as case. 
Research questions addressed are: (1) which methods of 
ADE assessment are used, (2) what is the range of non- 
serious ADEs captured for each method, (3) do the 
observational studies address known limitations of pre- 
approval trials regarding patient population and follow- 
up. 

Methods 

Search Strategy 

We conducted a systematic search of MEDLINE and 
EMBASE for observational studies reporting on ADEs in 
patients with diabetes, and published between January 1 
1999 and January 1 2009. We searched for papers using 
MeSH headings, subheadings and free-text terms related 
to the following domains: (1) "adverse events", and (2) 
"observational study design", and (3) "drug treatment" 
combined with "diabetes" (see Additional file 1 for 
detailed description of the search strategy). Using the 
boolean operator 'AND', only papers satisfying all three 
domains were included. 

Study Selection 

Observational studies, i.e. non-experimental studies 
where decisions regarding the prescription of drugs to 
each patient were made by their health care provider in 
every-day clinical practice, were included when they 
reported rates of non-serious ADEs in adult patients 
with type 2 diabetes mellitus treated with OAD. We 
excluded open-label extensions of clinical trials. Non- 
serious ADEs were defined as any unfavourable and 
unintended sign (including abnormal laboratory values) 
or symptom or disease that may present during treat- 
ment with a pharmaceutical product and which was not 
life-threatening, requiring hospitalization or resulted in 
significant disability or death. 

The first title and abstract screening was done by LH, 
excluding editorials, comments, notes, letters, rando- 
mized clinical trials (RCTs), case reports, and studies 
not including patients with diabetes or not including 



OAD (see also Figure 1 for exclusions). PD screened a 
10% sample which showed that LH had not excluded 
any potentially relevant studies. Screening of the 
remaining abstracts and full-texts was done by two 
reviewers independently. We restricted our selection to 
studies published in English, German, French, Spanish 
or Dutch language. 

Data Extraction 

Information was collected from the selected publications 
each by two reviewers (PD/LH, DD/LH or FHR/LH) 
using a standardized data extraction form. Data were 
extracted regarding methods used for assessing ADEs, 
the ADEs identified, inclusion and exclusion criteria of 
patient population, sample size, and duration of follow- 
up. In addition, we extracted data on study design and 
medications covered. Discrepancies in data extraction 
occurred in 3 cases regarding 'methods used for asses- 
sing ADEs', in 8 cases regarding 'sample size', and 9 
cases regarding 'duration of follow-up'. These discrepan- 
cies were often the result of unclear descriptions in the 
publications, and were solved by consensus based on a 
joint re-evaluation of what was described in the 
publication. 

Methods for ADE assessment 

ADE assessment in observational studies can be based 
on review of existing practice-based data, such as medi- 
cal records, laboratory reports, and administrative data, 
on surveillance by health care professionals (HCP) or on 
survey of patients [9,10,14]. Following this distinction, 
we defined the employed methods as: 1) medical record 
review, i.e. possible ADEs were collected from documen- 
tation or reports made by HCP in existing medical 
records; 2) solicited surveillance by HCP, i.e. requesting 
HCP to report possible ADEs either on Case Report 
Forms (prospective) or on socalled Prescription Event 
Monitoring forms (retrospective) [7]; 3) patient survey, 
including the use of open or closed patient question- 
naires, checklists or diaries; 4) administrative data, mak- 
ing use of diagnostic codes related to possible ADEs in 
administrative or claims data; 5) laboratory or clinical 
values indicating ADEs, including results of laboratory 
measurements and physical examinations such as weight 
or blood pressure; 6) non-specified methods. Reported 
ADEs were categorized on anatomy or pathophysiology 
level according to Common Terminology Criteria for 
Adverse Events (CTCAE v3.0) classification [15]. 

Patient population 

Based on the reported patient inclusion and exclusion 
criteria, we classified studies as: (A) restricting the 
patient population to lower risk patients, (B) restricting 
to higher risk patients, (C) applying restrictions needed 
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Total citations found 
10,125 

Identified from: 

IVIEDLINE: 3,584 
EMBASE: 6, 541 



Full-text review 
n=904 



Final set for data 
extraction 

n=64 including 
68 studies 



Excluded 
(based on title and 
abstract review) 
n=9,221 



Excluded 
n=840 



Reasons for exclusion: 

- No original studies 
(editorials, comments, 
notes, letters) 

- RCTs, case reports and 
case series 

- Not in diabetes patients 

- Not with oral 
antihyperglycemic drugs 



Reasons for exclusion: 

- No type 2 diabetes 
mellitus (sub) population 

- No rates reported on 
non-serious ADEs 

- Languages: Polish 
(n=5); Japanese (n=1), 
Norwegian (n=1) 

- No report of original 
study (systematic reviews, 
meta-analysis) 

- No observational study 



Figure 1 Study flow diagram. 



to achieve reliable outcome assessment, e.g. by exclud- 
ing patients with a condition or medication use at base- 
line which would confound the outcome, (D) no 
restrictions reported. 

Sample size and duration of follow-up 

We assessed the number of patients exposed to OAD, as 
well as the duration of their follow-up. For studies 
including more than one treatment group, we consid- 
ered the sample size of the largest group exposed to 
OAD treatment. For studies including a diabetic subco- 
hort, the overall number of exposed patients was consid- 
ered as the sample size. Based on recommendations 
from regulatory agencies for safety assessment 
[11,12,16,17], we categorized sample sizes into six levels: 
1) < 100 patients; 2) 100 to 299 patients; 3) 300 to 599 
patients; 4) 600 to 1499 patients; 5) 1500 to 5000 and 6) 
> 5000 patients. Duration of follow-up for cohort stu- 
dies was classified into: 1) <6 months; 2) 7-12 months; 
3) 13 to 24 months; 4) more than 2 years. 



Data Analysis 

Some publications reported on multiple studies with dif- 
ferent patient populations and methods. We conducted 
analysis at this study level. We present the type, median 
number and interquartile range (IQR) of ADEs at cate- 
gory level reported for the six different methods of ADE 
assessment. Sample size and duration of follow-up are 
also compared for the different ADE assessment meth- 
ods. We calculated the number of studies reaching regu- 
latory recommendations for pre-approval safety 
assessment of drugs intended for long-term treatment of 
non-life-threatening conditions, i.e. 100 patients exposed 
for a minimum of 1 year or 300-600 patients for 6 
months can be adequate to assess the pattern of ADEs 
over time [11,12]. 

Results 

The search resulted in 10,125 articles, out of which we 
selected 904 articles for full-text screening (Figure 1), 
resulting in 64 relevant articles reporting on 68 studies 
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(see Additional file 2 for a description of the included 
studies). 

Methods of ADE assessment 

The most commonly employed methods for assessing 
ADEs were based on laboratory/clinical values (n = 25), 
medical record review (n = 18), and solicited surveil- 
lance by HCP (n = 17) (Table 1). Surveillance by HCP 
was conducted prospectively using Case Report Forms 
in 12 studies, and retrospectively in 5 Prescription Event 
Monitoring studies. Among the 15 studies which used 
patient survey methods, 10 studies used a closed ques- 
tionnaire, including two validated questionnaires [18,19], 
one used a checklist [20], one used a semi-structured 
interview guide where patients could report any per- 
ceived ADEs [21], and one used a 16-item content-vali- 
dated questionnaire, containing closed and open-ended 
questions focusing among other issues on specific 
adverse events [22]. A patient diary was used in two stu- 
dies [23,24]. Administrative databases were used in 5 
studies, and in 7 studies, the data collection method was 
not fully specified. 

ADEs identified with different methods 

The largest range of ADEs was identified with solicited 
surveillance by HCP, yielding a median of 4 ADE cate- 
gories (Table 1). The range was even higher for retro- 
spective surveillance (median 7, IQR 4-9) in comparison 
to prospective surveillance (median 3.5, IQR 2-6). Medi- 
cal record review identified a median of 2 ADE cate- 
gories (Table 1), covering many different areas (Table 
2). Other specified methods assessed mostly 1 ADE 
category per study. Patient survey methods often 
focused on perceived hypoglycaemia or gastrointestinal 
ADEs (Table 2). Administrative databases were mainly 
used for cardiac ADEs, and laboratory/clinical values 
often included hepatic or metabolic problems or weight 
increase (Table 2). Four studies identified the same 
ADE, either hypoglycaemia or hepatic dysfunction, using 
more than one method, in particular a combination of 
laboratory values and other methods [25-28] . 



Patient population 

In 28 studies (41%), there were no specific limitations 
regarding the patient population included. In two studies 
(3%), no inclusion or exclusion criteria were specified 
[29,30]. Thirteen studies (19%) limited inclusion of 
patients to lower risk patients (category A) by including 
only patients with less severe diabetes [20,26,27,31-33] or 
patients on monotherapy [19,24,27,33-36], or OAD-naive 
patients [27,35] or by excluding high risk patients who 
failed previous therapy [37] or with multiple comorbidity 
[20,38,39]. Fifteen studies (22%) Umited the inclusion to 
more complicated cases (category B), such as inadequately 
controlled by or not tolerating previous medication 
[40-45], receiving combination treatment [46-48] or insu- 
lin [21,23,45,49] or treated with maximum dose of medica- 
tion [50]. Furthermore, 18 studies (27%) excluded patients 
based on the presence at baseline of the outcome or a con- 
dition that could influence the outcome 
[18,24,25,33,37-39,47,51-55], non-availability of measure- 
ments and/or clinical visits [35,37,46,47,50,54,56,57], 
inability to fill in questionnaires (category C) [18,21,46,56]. 

Sample size and duration of follow-up 

Studies using patient survey methods, medical record 
review, or laboratory data often included less than 300 
patients (Figure 2). A sample size of equal or more than 
1500 was achieved by all studies using administrative data- 
bases, and in many studies using solicited surveillance by 
HCP. Overall, the follow-up period did not exceed one 
year in 77% of the cohort studies. Longer follow-up peri- 
ods were mostly seen in studies using administrative data 
or laboratory/ clinical values. Evaluation of sample size and 
follow-up jointly showed that all 3 cohort studies using 
administrative data exceeded the requirements of the 
guidelines for pre-approval safety assessment, whereas this 
was the case in less than a quarter of the studies using any 
of the other specified methods (Table 3). 

Discussion 

Commonly used methods for assessing non-serious 
ADEs in patients with diabetes were based laboratory or 



Table 1 Median number and interquartile range (IQR) of different ADE categories identified for studies using different 
assessment methods 





Number of studies^ 


median number of ADE categories (IQR) 


References 


Method of ADE assessment 


Medical record review 


18 


2 (1-3) 


[22,25,34,35,38,40,41,49-52,78-83] 


Surveillance by HCP 


17 


4 (2-7) 


[23,29,30,42,43,84-95] 


Patient survey 


15 


1 (1-2) 


[18-24,26,31,40,44,46,53,56] 


Administrative data 


5 


1 (1-1) 


[33,47,54,96,97] 


Laboratory/clinical values 


25 


1 (1-2) 


[25-28,32,35-37,39-41,44,45,50,55,57,80-83,98-101] 


Non-specified 


7 


2 (1-10) 


[27,28,36,48,99-101] 



* Total exceeds 68 since one study may use several methods 
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Table 2 Types of ADEs reported at category level for studies using different assessment methods (number of studies 


presented in table) 








Adverse events at CTCAE category 


Medicalrecord HCP surveill- 


Patient Admini-strative Lab/clinical 


Non 


level 


review a nee 


survey data values 


specified 


Gastrointestinal 


9 14 


3 


5 


Neurology 


3 6 


1 


3 


Cardiac General 


9 9 


4 1 


4 


Blood/Bone Marrow 


2 4 


5 


1 


Pulmonary/Upper Respiratory 


1 2 




2 


Hepatobiliary/Pancreas 


3 7 


1 11 


2 


Auditory/Ear 


1 




1 


Ocular/Visual 


1 






Dermatology/Skin 


1 4 




3 


Musculoskelal/Soft Tissue 


1 




1 


Renal/Genitourinary 


1 1 


2 


2 


Constitutional symptoms: 








- weight 


6 


12 




- other 


1 3 


1 




Pain 


3 7 


2 




Endocrine 


1 






Infection 


1 




3 


Allergy/Immunology 


1 






Sexual/Reproductive Function 


1 






Metabolic: 








- hypoglycaemia 


7 7 


8 3 


5 


- other 


4 


1 7 


1 


General ADEs^olerability* 


3 12 


3 3 


5 


CTCAE Common Terminology Criteria for Adverse Events vB.O; ^ not categorized 



approval requirements regarding sample size and dura- 
tion of follow-up. 

Solicited surveillance by health care providers, using 
either prospective or retrospective data collection, 
revealed the largest diversity of ADEs, indicating that 
doctors register more events on such forms than in rou- 
tine medical records. This is in line with previous find- 
ings that medical record review, although broadly used 
for assessing ADEs, results in incomplete findings 
[11,58]. Since there is no systematic documentation of 
ADEs in medical records, partly due to limitations of 
the documentation systems [59,60], review of such 
records lacks a standardized and reliable method to 
search for ADEs [61]. For non-serious, symptomatic 
ADEs the incomplete documentation of adverse events 
in medical records is even more the case when such 
ADEs do not warrant immediate action [1,62]. Prescrip- 
tion Event Monitoring studies, which make use of an 
open question to report all events that occurred during 
drug use on special forms, or prospective studies using 
prespecified Case Report Forms may solve this problem. 

Patient reports can be of great value for ADE assess- 
ment because of the differences between reports from 
patients and health care providers [4,63-66]. Patients are 



clinical values, medical record review or solicited sur- 
veillance by HCP. The latter method identified the 
broadest range of ADE categories. Patient survey meth- 
ods were used in 22% of the studies, and often focused 
on a limited range of ADEs, such as hypoglycaemia or 
gastrointestinal ADEs. The patient population was 
restricted to a lower risk population in a fifth of the stu- 
dies. Less than one-third of studies exceeded pre- 
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■ W r 
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Medical HCP Patient Admininstra- Lab/clinical Non- 
record surveillance survey tive data values specified 

I □<1 00 1100-299 1300-599 □ 600-1499 □ 1500-5000 ■>5000 patients | 

Figure 2 Sample size included in studies using different 
assessment methods. 
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Table 3 Number of cohort studies for each assessment method where sample size and follow-up period exceed 
regulatory recommendations for pre-approval safety assessment 

Regulatory recommendations [11,12] 

Method of ADE assessment Total number of cohort studies > 100 patients > 12 months > 300 patients > 6 months 



Medical record review 17 

Surveillance by HCP 17 

Patient survey 6 

Administrative data 3 

Laboratory/clinical values 22 

Non-specified 7 

Total 71 



* Total exceeds 68 since one study may use several methods 

a helpful source for the identification of many sympto- 
matic ADEs, such as dizziness, malaise, fatigue, sexual 
function disorders, and pain [67-69]. Surprisingly, we 
found that patient survey methods were used in a rela- 
tively small number of studies, and moreover, often lim- 
ited in their focus. Although comprehensive 
questionnaires have been developed to assess patient- 
perceived ADEs [70,71], such questionnaires were not 
used in observational studies for diabetes treatment. 

Laboratory values may have a limited value for asses- 
sing non-serious ADEs, since mainly hepatic and meta- 
bolic problems were identified by these methods. This is 
in contrast with previous estimates that more than half 
of the ADEs can be detected by biochemical tests [72]. 
Administrative databases are also limited regarding the 
types of ADEs that can be identified. Such databases can 
be useful for assessing ADEs leading to hospitalization 
but have less value for assessing non-serious ADEs. 
Diagnostic administrative coding is likely to be both 
incomplete and unspecific for detecting non-serious 
ADEs [73], because these ADEs do not always call for a 
documented action from the health care provider [1,62]. 
Currently, European Medicines Agency regulators work 
on strengthening this source of information by establish- 
ing a European Network of Centres for Pharmacovigi- 
lance and Pharmacoepidemiology [74]. 

Combining methods for ADE assessment could 
address some limitations seen with all methods leading 
to under- or overreporting. ADEs which are likely to be 
underreported because of improper registration or cod- 
ing in medical records might be complemented by 
laboratory values [73]. The same applies to doctor and 
patient reports that may complement each other [75]. In 
our review, however, only a four studies identified the 
same ADE using a combination of methods. 

Observational post-marketing studies can provide 
additional information on ADEs when sufficient num- 
bers of patients are being followed in daily practice, 
including those with higher risks, more comorbidity, 
concomitant drugs, and longer disease duration. The 



0 4 

0 4 

0 1 

0 3 

3 3 

2 1 

5 17 



majority of studies in our review included such patient 
populations, thus adding valuable information on ADEs 
in patient groups underrepresented in pre-approval 
trials. The number of patients included and the duration 
of follow-up, however, showed similar limitations as 
pre-registration trials, and the majority of studies did 
not go beyond the pre-approval recommendations for 
safety assessment of diabetes medication. Because of 
workload, long follow-up for large numbers of patients 
can be problematic in studies where the patients or the 
health care providers need to provide the information. It 
is less problematic when data can be collected from 
existing databases. 

Our study has some limitations. It has previously been 
recognized that searching the literature for studies 
reporting on drug safety is difficult [76,77], and also 
indexing of observational studies may not be as robust 
as of RCTs. We therefore used a broad search strategy 
to identify possibly relevant studies. Second, the results 
are based on studies conducted in diabetes patients 
using OADs. For other therapeutic areas and other 
drugs, results may be different. Third, we used the 
CTCAE v3.0 classification to define ranges of ADEs 
identified by different methods. Although the CTCAE 
categories are quite similar to the primary system organ 
classes in the MedDRA hierarchy, minor differences in 
ranges may occur when using this alternative classifica- 
tion system. Finally, we encountered several problems 
regarding unclear or incomplete reporting. Although it 
was not our aim to evaluate studies on the quality of 
reporting, and we did not exclude studies on these 
grounds, we observed that information on, for example, 
exclusion criteria and response rates was often lacking. 

Conclusion 

The current set up of ADE assessment in post-market- 
ing studies is not adequate for countering limitations 
acknowledged in pre-approval trials. The assessment of 
non-serious ADEs is limited by the choice of methods. 
Many observational studies rely on methods that are 
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inadequate for identifying all possible ADEs. Patient sur- 
vey methods are underutilized, and there is a lack of 
studies that try to combine different methods to assess 
ADEs. This implies that these studies will not provide 
sufficient information about ADEs to clinicians and 
patients. Better protocols are needed on how to assess 
adverse drug events not only in clinical trials but also in 
observational studies. 
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